Skip to content

air: show the Databricks AI environment in air get - #7637

Merged
dimk0 merged 5 commits into
mainfrom
cnxt-2641-air-get-ai-environment
Nov 17, 2026
Merged

dimk0 merged 5 commits into
mainfrom
cnxt-2641-air-get-ai-environment

Conversation

@dimk0

@dimk0 dimk0 commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Changes

  • air get text output now shows the environment, for example databricks_ai_v6, for runs submitted with environment.version: databricks_ai_v*, instead of Environment N/A. The air list info pane renders through the same code and is fixed too.
  • air run submits those versions as spec.base_environment = workspace-base-environments/<version> with no environment_version. The Environment cell now shows environment_version when it is set, else the workspace base environment ID with that prefix stripped. The prefix is the constant workspaceBaseEnvironmentPrefix, shared with buildRunEnvironment and convert-to-dabs. Any other workspace base environment, such as a custom dbe_<uuid>, shows its ID the same way.
  • air get -o json is unchanged: since air: Include run details in air get JSON #7624 it reports environment_version and the raw base_environment.
  • The Environment cell is passed through stripControl, like the other server-supplied cells, so escape sequences in a base environment name cannot reach the terminal.

Why

Every run on the Databricks AI environment, which most AI Runtime cookbook recipes use, showed Environment N/A. A real jobs/runs/get response for such a run, from air run with environment.version: databricks_ai_v6, returns "environments": [{"environment_key": "default", "spec": {"base_environment": "workspace-base-environments/databricks_ai_v6"}}].

Tests

  • TestAiRuntimeEnvironment: for a numeric environment_version, a databricks_ai_v6 base environment and a custom dbe_<uuid> base environment, checks the raw values read from runs/get and the Environment cell text.
  • TestEnrichRunDetails: the base-environment case now expects the Environment cell databricks_ai_v5 instead of N/A; its raw base_environment value is unchanged.
  • New acceptance test acceptance/air/get-databricks-ai: air get text output for a databricks_ai_v6 run.
  • TestRenderFieldsStripsControlCharacters and the acceptance/air/control-chars test now put an escape sequence into the environment and assert clean output.
  • Each new test fails without its fix. go test ./cmd/air/... and go test ./acceptance -run 'TestAccept/air/' pass; ./task lint-q, ./task fmt-q and ./task check-changelog are clean. Unit tests ran with DATABRICKS_WORKSPACE_ID unset: TestLogsCommandRepeatsRunLinkAfterStreaming also fails on main when that variable is set.

This pull request and its description were written by Isaac.

dimk0 and others added 2 commits November 14, 2026 23:42
`air run` submits a databricks_ai_v* version as spec.base_environment
(workspace-base-environments/databricks_ai_v6), but `air get` and the
`air list` info pane read only spec.environment_version, so Environment
showed N/A. Strip the shared prefix and show the configured version.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 14, 2026
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 2dccfb2f8 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

2 tests: regression=2 unreleased=0 coverage=0 failing=0
test branch main (a96363b) latest (v1.26.0)
TestAccept/air/control-chars ✅ ❌ ❌
TestAccept/air/get-databricks-ai ✅ ❌ ❌
TestAccept/air/control-chars: branch ✅ | main (a96363b) ❌ | latest (v1.26.0) ❌

main (a96363b):

acceptance_test.go:1380: Diff:
        --- air/control-chars/output.txt
        +++ /tmp/TestAcceptaircontrol-chars457862426/002/output.txt
        @@ -15,7 +15,7 @@
         │  MLflow Run    my-run                                          │
         │  User          user@example.com                                │
         │  Accelerators  1x A10                                          │
        -│  Environment   databricks_ai_v6                                │
        +│  Environment   N/A                                             │
         │                                                                │
         ╰────────────────────────────────────────────────────────────────╯
         
        
FAIL

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- air/control-chars/output.txt
        +++ /tmp/TestAcceptaircontrol-chars3665722735/002/output.txt
        @@ -9,13 +9,14 @@
         │  Status        ● SUCCESS                                       │
         │  Submitted     2023-11-14 22:13 UTC                            │
         │  Retries       0                                               │
        -│  Max Retries   N/A                                             │
        +│  Max Retries   0                                               │
         │  Duration      12s                                             │
        -│  Experiment    my-exp                                          │
        -│  MLflow Run    my-run                                          │
        -│  User          user@example.com                                │
        +│  Experiment    my�]0;pwned�-exp                                          │
        +│  MLflow Run    �]0;pwned�my�[2J-run
        +                                          │
        +│  User          user�[31m@example.com                                │
         │  Accelerators  1x A10                                          │
        -│  Environment   databricks_ai_v6                                │
        +│  Environment   N/A                                             │
         │                                                                │
         ╰────────────────────────────────────────────────────────────────╯
         
        @@ -26,7 +25,8 @@
         
         >>> [CLI] air list --all-users
            Run ID  Experiment  Status     Started              Duration  Progress  MLflow                  User              Accelerators
        -   123     my-exp      ● SUCCESS  [TIMESTAMP]       12s         -  my-run                  user@example.com  1x A10
        +   123     my�]0;pwned�-exp      ● SUCCESS  [TIMESTAMP]       12s         -  �]0;pwned�my�[2J-run
        +                  user�[31m@example.com  1x A10
         
         === list (json): strings are passed through as the server sent them
         
        @@ -42,12 +42,7 @@
                 "user": "user\u001b[31m@example.com",
                 "status": "SUCCESS",
                 "started_at": "[TIMESTAMP]",
        -        "is_sweep": false,
        -        "experiment_name": "my\u001b]0;pwned\u0007-exp",
        -        "duration_seconds": 12,
        -        "accelerator_type": "GPU_1xA10",
        -        "num_accelerators": 1,
        -        "dashboard_url": "[DATABRICKS_URL]/jobs/runs/123?o=[WORKSPACE_ID]"
        +        "is_sweep": false
               }
             ]
           }
        
FAIL
TestAccept/air/get-databricks-ai: branch ✅ | main (a96363b) ❌ | latest (v1.26.0) ❌

main (a96363b):

acceptance_test.go:1380: Diff:
        --- air/get-databricks-ai/output.txt
        +++ /tmp/TestAcceptairget-databricks-ai1227633094/002/output.txt
        @@ -13,7 +13,7 @@
         │  MLflow Run    N/A                                             │
         │  User          user@example.com                                │
         │  Accelerators  1x A10                                          │
        -│  Environment   databricks_ai_v6                                │
        +│  Environment   N/A                                             │
         │                                                                │
         ╰────────────────────────────────────────────────────────────────╯

latest (v1.26.0): same output as main (a96363b)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 2dccfb2

Run: 44019892815

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 17 4:15
💚​ aws-windows-latest-4core-16gb 1 272 15 3:55
💚​ azure linux-2core-8gb 1 269 17 4:05
💚​ azure-windows-latest-4core-16gb 1 271 15 3:58
💚​ gcp linux-2core-8gb 1 270 17 4:47
💚​ gcp-windows-latest-4core-16gb 1 272 15 4:09
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:04 gcp-windows-latest-4core-16gb TestAccept
3:53 azure-windows-latest-4core-16gb TestAccept
3:49 aws-windows-latest-4core-16gb TestAccept

dimk0 and others added 2 commits November 15, 2026 22:12
- Strip control characters from the Environment cell, as for the other server-supplied cells, and cover it in the control-character tests.
- Reword the base environment comments: any workspace base environment shows its ID, not only databricks_ai_v*; pin that with a dbe_ test row.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@dimk0
dimk0 marked this pull request as ready for review November 16, 2026 03:50
@dimk0
dimk0 requested review from a team as code owners November 16, 2026 03:50
@dimk0
dimk0 requested a review from shreyas-goenka November 16, 2026 03:50
Co-authored-by: Isaac <no-reply@databricks.com>
@dimk0
dimk0 enabled auto-merge November 17, 2026 04:53
@dimk0
dimk0 added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit f3d4425 Nov 17, 2026
28 checks passed
@dimk0
dimk0 deleted the cnxt-2641-air-get-ai-environment branch November 17, 2026 05:20
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: f3d4425

Run: 44023476482

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 9 1 12 1613 1336 223:44
💚​ aws-windows-latest-4core-16gb 13 1541 1363 219:55
❌​ azure linux-2core-8gb 9 1 4 12 1589 1344 227:11
🔄​ azure-windows-latest-4core-16gb 4 11 1519 1371 233:26
❌​ gcp linux-2core-8gb 9 1 2 12 1598 1342 235:52
🔄​ gcp-windows-latest-4core-16gb 4 11 1526 1369 240:17
30 interesting tests: 11 flaky, 9 FAIL, 9 RECOVERED, 1 KNOWN
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🟨​ TestAccept 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/config-remote-sync/multiple_resources ✅​p 🙈​s 🔄​f 🙈​s ✅​p 🙈​s
🔄​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true ✅​p 🔄​f ✅​p
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/deploy/spark-jar-task ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/deploy/spark-jar-task/DMS=true ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/invariant/no_drift ✅​p ✅​p ✅​p 🔄​f 🔄​f 🔄​f
🔄​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=vector_search_index.yml.tmpl/READPLAN=1 ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=experiment.yml.tmpl/READPLAN=1 ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=pipeline_allow_duplicate_names.yml.tmpl/READPLAN=1 ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
💚​ TestAccept/bundle/resources/apps/lifecycle-started 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS= 💚​R 💚​R 💚​R 🔄​f 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=true 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
12:50 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:07 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:08 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:43 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:21 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:11 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:10 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:37 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:33 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:32 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
8:31 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:30 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:28 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:17 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:46 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:43 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:27 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:26 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:21 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
7:13 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:10 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:07 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:01 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:56 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:45 aws linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=true
6:43 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=
6:40 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:38 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
6:35 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:26 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:23 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:19 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:19 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:14 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:05 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
5:59 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:48 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:44 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:38 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:28 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
5:25 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
5:22 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
5:07 gcp linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=
5:05 azure-windows-latest-4core-16gb TestAccept/bundle/resources/job_runs/wait/DMS=
5:01 aws linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=true
5:01 azure linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_run.yml.tmpl/READPLAN=
4:54 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
4:51 azure-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
4:51 azure-windows-latest-4core-16gb TestAccept/bundle/deploy/spark-jar-task/DMS=true
4:46 gcp linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants